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@ Coordination of test sequences to be run si- 
multaneously by individual testers for conformance 
testing of multi-user systems to their specification is 
achieved in accordance with the present method. In 
order to utilize the present invention, it is necessary 
to generate a test sequence, desirably, one of mini- 
mum cost, which covers all aspects of the specifica- 
tion for the multi-user system under test. For many 
standard test sequence generation techniques, it is 
preferable to represent the multi-user system as a 
finite state machine. The test sequence is then di- 
vided into several different test sequences conre- 
sponding to individual testers wherein each of the 
different test sequences can be run simultaneously 
by individual testers. Then, the separate test se- 
quences are inspected to Identify instances where 
coordination is required among the various multiple 
testers which are expected to run the tests simulta- 
neously. After the coordination instances are iden- 
tified, additional instructions coordination primitives 
are inserted into the original test sequence for each 
tester to permit effective coordination between all 
testers. Coordination signalling is perfonmed on an 
out-of-band communication media between the test- 
ers. 
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COORDINATION METHOD FOR MULTI-USER SYSTEM CONFORMANCE TESTING 



Technical Field 

This invention relates to testing conformance to 
a prescribed specification or standard for mutti-user 
systems. 



Background of the Invention 

Diverse systems such as digital communication 
switches. PBXs, implementations of high-layer 
communication protocols, and VLSI circuits can be 
classified as multi-user systems because each en- 
tity offers services to more than one user simulta- 
neously. The tennn "user" is understood to be 
contQxt-sensitive comprising terminals, computers, 
chips, drcuit elements, or telephone equipment, for 
example. Testing such systems to ensure that each 
conforms to a standard specification continues to 
be a challenging task because of the complexity of 
the services each offers. Since multi-user systems 
and the services provided thereby involve differing 
numbers of users, conformance testing requires 
several test systems to be invoked for effective and 
thorough testing to be accomplished. For example, 
consider the voice conference service offered by a 
telecommunication switch where several users can 
•simultaneously talk to each other once a confer- 
ence call is set up. At the present time, testing 
such a sendee for its confomiance to the specifica- 
tion requires several manual testers or craftsper- 
sons each testing a different leg of the conference 
call setup. Such a procedure is obviously complex, 
cumbersome, and generally ineffective for fully 
testing system conformance and operability In light 
of a prescribed standard or specification. 

Recently, several methods have gained interest 
in replacing manual techniques with automated 
techniques for test sequence generation. See, for 
example. U.S. Patent 4,692,921 and U.S. Patent 
application Serial No. 362,724 (A Dahbura Case 3- 
5-1). While these techniques are useful for single 
use systems, there are several difficulties in auto- 
matically generating test sequences for multi-user 
systems. Complexity of the systems and services 
prevent the use of such generated test sequences. 
Of course, the methods can be applied to generate 
test sequences for each individual tester in a multi- 
user system. However, there is no procedure for 
testing the multi-user services where it is neces- 
sary to decide which tester will run a particular test 
and when the tester will run that test. Even though 
this problem has been recognized by industry ex- 
perts, it has been observed that, at ttiis time, co- 
ordination procedures to allow multi-user system 



testers to run simultaneously in a synchronized 
fashion remain an unsolved problem. See, for ex- 
ample, R.J.Linn, IEEE J. on Selected Areas in 
Communications, Vol. TTo. 7, pp. 1143-58 

5 (September 1989). 

Without tiie test coordination procedures, test 
laboratories are forced to employ inefficient and 
costly procedures in order to conduct multi-user 
systems test. For example, a single tester is placed 

70 in a multi-user environment whereas the remaining 
would-l)e testers are replaced by standard user 
equipment which is incapable of testing and verify- 
ing results. It is understood that a tester is typically 
a hardware, or software, or combined hardware and 

76 software system implemented by a test ialwratory 
for only testing purposes. It is capable of generat- 
ing unexpected and illegal behavior and timeouts 
that may be caused by a user in addition to the 
expected behavior. Furthermore, a tester is gen- 

20 erally equipped to analyze, in detail, any response 
received from the multi-user system under test, 
thereby, making the tester capable of detecting 
even small errors. By contrast, standard user 
equipment is generally a device implemented by a 

25 manufacturer for actual use in a mutti-user system. 
The standard user equipment cannot create unex- 
pected and illegal user behavior; it cannot cause 
timeouts^ and it cannot analyze the the response of 
ttie multi-user system in detail since it is built to 

30 generate standard default actions only. In one ex- 
ample, testing a digital telecommunication switeh 
for conference service without test coordination 
procedures is presently handled by a single spe- 
cialized tostor and several commercially available 

05 telephones. Coverage of testing for this example is 
not be complete since the standard telephone 
equipment cannot generate certain unexpected or 
illegal behavior tiius leaving ttiose aspects of the 
multi-user system behavior untested. In such a 

40 minimal test environment, the effectiveness of the 
testing is severely limited. 



Summary of the Invention 

Coordination of test sequences to be run si- 
multaneously by Individual testers for conformance 
testing of multi-user systems to their specification 
is achieved in accordance with the present method. 
50 In order to utilize tiie present invention, it is neces- 
sary to generate a test sequence, desirably, one of 
minimum cost, which covers all aspects of the 
specification for the multi-user system under test 
For many standard test sequence generation tech- 
niques, it is preferable to represent the multi-user 
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system as a finite state machine. The test se- 
quence is then divided into several different test 
sequences corresponding to individual testers 
wherein each of the different test sequences can 
be run simultaneously by individual testers. Then, 5 
the separate test sequences are inspected to iden- 
tify instances where coordination is required among 
the various multiple testers which are expected to 
run the tests simultaneously. After the coordination 
instances are identified, additional instructions co- to 
ordination primitives are Inserted into the original 
test sequence for each tester to permit effective 
coordination between all testers. Coordination sig- 
nalling is performed on an out-of-band communica- 
tion media between the testers, thereby, providing ts 
a very flexible testing environment. Also, there Is 
no central mechanism for controlling all testers In 
this method which allows testers to reside in re- 
mote sites. 



Brief Description of the Drawing 

A more complete understanding of the inven- 
tion may be obtained by reading the following 25 
description of a specific illustrative embodiment of 
the invention in conjunction with the appended 
drawing in which: 

RG. 1 is a finite state machine representation of 
an exemplary multi-user system; 30 
FIG. 2 is a flow chart diagram of the method 
• steps in accordance with tfie principles of the 
invention; and 

FIG. 3 through 11 show test sequence tables 
derived at various steps in accordance with the 35 
present inventive method. 



Detailed Description 

40 

A specification for an entity such as a hardware 
device or a software application or a communica- 
tions protocol can be described as an abstract 
model called a finite state machine. The finite state 
machine consisting of a finite number of states, a 45 
defined set of inputs or stimuli which can be ap- 
plied to the finite state machine, and a defined set 
of observable outputs generated by the finite state 
machine. The finite state machine changes from 
one state to another when an input or stimulus is so 
applied to the machine. A state is defined as a 
stable condition in which the entity or finite state 
machine rests until the next stimulus or input is 
applied. Each input also causes the finite state 
machine to generate an observable output. RG. 1 55 
shows a graph representation of a finite state ma- 
chine for a multi-user system involved with ISDN 
basic rate interface supplementary service (call 



holding) which is to be provided by switching ma- 
chines. The finite state machine shown in FIG. 1 is 
deterministic because a particular input symbol 
causes at most one transition from each state in 
the machine. 

As shown In FIG. 1 , the finite state machine Is 
represented as and its behavior are described by a 
directed graph called a state diagram wherein a 
node or vertex of the graph (shown as a circle in 
FIG. 1) conresponds to a state of the finite state 
machine and wherein a directed edge between two 
nodes or a node and itself (shown as an arrow 
between two circles in FIG. 1) corresponds to a 
transition from one state to another. The directed 
graph representation of a finite state machine pro- 
vides an easy way to visualize the operation of the 
finite state machine and, thereby, the conrespond- 
ing entity. Each directed edge of the graph is 
labeled both by the particular input symtwl which 
causes the state transifion and by the observable 
output symbol generated thereby. 

To model the fact that more than one testing 
entity or tester interacts with the multi-user system 
by sending signals to or receiving signals from the 
testing entities, edge lab>els are applied to the 
directed graph according to the following notation: 
A ? input) / B ! outputj 

where the multi-user system receives an input 
called inputi fi'om a testing entity called Tester A 
and sends an output called outputj to a testing 
entity called Tester B. Testing entities A and B are 
generally different testers, but they may be the 
same testing entity. It is assumed that all testing 
entities communicating with a multi-user system 
are external to the multi-user system implementa- 
tion. That is. inputs sent to a multi-user system by 
testing entities are controlled or generated exter- 
nally with respect to the multi-user system and 
outputs sent by a multi-user system to the testing 
entity or testing entities are observable or verifiable 
externally with respect to tiie multi-user system. 
Such an assumption includes almost all practical 
implementations of multi-user systems In the ares 
of digital communications and VLSI circuit design. 

As stated above, the actions of sending an 
output to a testing entity and receiving an input 
fi'om a testing entity are denoted as ? and /, 
respectively. In FIG 1. the edge from state NO to 
state N1 which is labeled as 
User1?setup/User2ldialtone. conresponds to the fol- 
lowing action: When the multi-user system is in 
state NO, it receives an input called 'setup'' 
from User 1, sends an output called "dial tone" 
to User U and moves to state Nh In this context, 
the tenm "User" is used synonymously and inter- 
changeably with the tenn "Tester". 

In a label on an edge of the directed graph of 
the finite state machine for the multi-user system, 
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there can only be one input received from multiple 
testing entities. However, the multi-user system can 
generate outputs which are to be sent to more than 
one testing entity. In this case, an edge is shown 
with the following notation: 
A ? inputi / B ! outpu^ . C ! outputu • • • 
where upon receipt of an input called inputi from 
testing entity A, the multi-user system sends out- 
puts called outpu^ to testing entity B and output^ to 
entity C. Each output is separated by commas. 
There cannot be two outputs to the same testing 
entity generated by a single Input For example, in 
RG 1, the edge from state N1 to state N2 labeled 
as 

User! ! digits / User1 ! call__proc , User2 ! setup 
is interpreted as follows: the multiuser system 
receives the input called digits from Userl as 
state N1 and moves to state N2 while sending 
caii^proc output to Userl and setup to User2, 

Testing a mdrti-user system implementation for 
conformance to its specification requires multiple 
testers to be run simultaneously in a coordinated 
fashion. In other words, all testing entities commu- 
nicating with the multi-user system are required to 
send inputs to and expect outputs from the multi- 
user system at predetenmined time instances. In 
order to realize a multiple tester environment it is 
important to understand tiiat tests that are run 
simultaneously by the various testers should cover 
substantially every state transition of the finite state 
machine representing the multi-user system speci- 
fication; tiie' total number of tests to be run by the 
various testers should be minimal since the testing 
time for complex protocols and the like may other- 
wise require infeasibly long testing times; since a 
multi-user system specification is typically given as 
a combination of Input or output interactions with 
various testing entities with which the multi-user 
system communicates, the tests defined for a 
multi-user system implementation should be de- 
fined for individual testers, each of which repre- 
sents a testing entity tiiat communicates with fhe 
multi-user system; and each tester should be syn- 
chronized with each other in order to test an multi- 
user system implementation conrectiy. Th latter 
consideration requires a mechanism called the test 
coordination procedure among the various testers 
which is described and implemented herein in ac- 
cordance with the prindples of the invention. 

The method which realizes the above-men- 
tioned considerations for generating multiple tester 
environment for an multi-user system is shown in 
FIG. 2. For the first and second considerations 
disclosed above, the method described herein- 
below may use the procedure disclosed in U. S. 
Patent 4,692,921 for generating test sequences for 
a specification represented by a finite state ma- 
chine. The prefenred procedure given above gen- 



erates tests tiiat require minimum run-time while 
completely covering every state transition defined 
for a finite state machine. For the third and fourth 
considerations disclosed above, a procedure is in- 

5 troduced to automatically divide tiie generates test 
sequence among the several testers and to detect 
the instances in each of the divided test sequences 
where coordination among tiie testers Is needed. 
FIG 2 depicts the steps of tfie method to 

70 generate coordinated test sequences for the con- 
formance testing of a multi-user system in accor- 
dance with tiie principles of the present invention. 

In the first step, step 10, a test sequence for a 
multi-user system is generated based on its finite 

15 state machine model. The test sequence com- 
prises consecutive inputs and outputs to be sent to 
and received from tfie multi-user system imple- 
mentation under test. The finite state machine 
model represents the behaviour of the multi-user 

20 system which is communicating with several test- 
ing entities under various circumstances including 
expected, unexpected and erroneous behavior of 
the testing entities. Therefore, it is important for the 
test sequence that all state transitions defined in 

25 the finite state machine model are covered. The 
method described herein assumes that such a 
model is available for a multi-user system. The test 
sequence generated in the first step is called the 
"initial test sequence" in order to distinguish it from 

30 the coordinated test sequences that will be gen- 
erated in the latter steps of the inventive method. 

An exemplary minimal initial test sequence for 
tiie diagram shown in FIG. 1 is shown in FIG. 9 and 
continued in RG. 10. The format of the initial test 

35 sequence In generalized form is shown in the table 
of FIG. 3. In RG. 3. tfie field called TEST STEP 
gives the order in which the inputs are applied to 
tiie multi-user system under test p.e., TEST STEP i 
is followed by TEST STEP i + 1 and i+2 and so 

40 on). The fields called CURRENT STATE and NEXT 
STATE indicate the cunrent and expected new state 
of the multi-user system implementation, respec- 
tively. The input to be sent to the multi-user sys- 
tem from tiie testing entity (including the name of 

45 tiie entity tiiat generates tiie input) and tiie ex- 
pected output or outputs to be sent to the testing 
entity or entities (including the name or names of 
tiie entities that are expecting to receive the out- 
puts) are given in the fields called MSG TO MUS 

50 and MSG FROM MUS, respectively, where MUS 
stands for multi-user system. By refem'ng to FIG. 3, 
it can be seen that, during TEST i, the multi-user 
system implementation under test is in state 
STATEj. A testing entity called A sends an input 

55 called inputi to the multi-user system and the multi- 
user system is expected to send outputs called 
outputm and outputn to the testing entities called B 
and C, respectively, (or more entities as defined in 
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the multi-user system specification), The new state 
of the multi-user system t)ecomes STATEk. After 
TEST i is executed, the initial test sequence contin- 
ues with the execution of TEST i + 1 where the 
current state of the multi-user system is STATEk. 

During testing, all outputs generated by the 
multi-user system implementation under test 
should have been be by the multi-user system 
specification for every test of the sequence. Other- 
wise, the implementation may experience failure for 
that test 

In the second step, step 20. of the method in 
FIG. 2. the initial test sequence generated in the 
first step is divided among various testers to which 
each particular test step corresponds. Testers si- 
multaneously run to execute their individual tests 
that are determined for each of them at this step. In 
order for proper operation of the method and in 
addition to communicating with the multi-user sys- 
tem implementation\ it is desirable for testers to 
communicate with each other by means of an 
extemal media which, depending on the testing 
environment, may be a telephone line, a software 
link (i.e.. a virtual link) or a cable. Such an inter- 
tester communication arrangement is typically 
called out-of-band communication. The characteris- 
tics of tiie test environment determine the selection 
for the simplest and the most appropriate out-of- 
band communication media 

During testing, each entity (user) defined in the 
finite state machine model of a multi-user system 
is replaced wrth an individual tester so that the 
correctness of the interactions between the entities 
and the multi-user system implementation can be 
tested. Therefore, the number of testers needed to 
test a multi-user system implementation is equal to 
the number of entities (users) witii which the multi- 
user system communicates. User entities that a 
multi-user system communicates with are specified 
in the specification of the multi-user system, and 
therefore, are included In the edge labels of its 
finite state machine model. This information is also 
included in the initial test sequence, since the initial 
test sequence generated at the previous step is 
based on tiie finite state machine model and, there- 
fore, the specification of a multi-user system. 

From the table in RG. 3. it is detenmined that 
three entities and. therefore, that three testers, 
called TESTER A, B and C. are needed in order to 
run TEST STEP i of ttie initial test sequence. 
Between tiie multi-user system implementation and 
the tiiree testers, a message exchange takes place 
during TEST i as shown in FIG, 4. FIG. 4 shows 
the assignment and division of particular test steps 
among tiie various test entities. During TEST STEP 
i. TESTER A sends inputi to the multi-user system 
implementation under test (as shown in the second 
column of tiie test sequence table of TESTER A) 



and TESTER B and C expects outputs called out- 
putm and outputn. respectively, from the multi-user 
system implementation (as shown at tiie tiiird col- 
umn of each tester table). Since the testers need 

5 not know what state the implementation is in. this 
Infomnation Is not included in the tester tables (note 
tiiat, if the implementation moves to a wrong state, 
ttie output will be different tiian what is specified in 
tiie specification for the next test and the error will 

TO be caught eventually by the test sequence gener- 
ator). At the end of the third step of the method, 
every test of the initial test sequence is divided into 
the exchange of inputs and outputs between the 
multi-user system implementation under test and 

75 ttie individual testers such that TEST STEP i of 
initial test sequence corresponds to TEST STEP i 
for every individual tester. During testing, every 
tester must execute TEST STEP i (for i = 1.2.3...x 
for an initial test sequence of length x) simuita- 

20 neously. ^ 

In order to perform tiie tests correctly. TEST 
STEP i of each tester must be executed simulta- 
neously among all testers. Given that each tester 
may reside in remote sites, a coordination mecha- 

25 nism synchronizes the testers. While the testers 
may not necessarily be physically remote, they 
may run in the same system however each may be 
residing in an independent software process and, 
therefore, be considered as remote. 

30 In the third step, step 30. of the method, the 

tests of each tester for which synchronization and 
coordination is needed "are determined. Coordina- 
tion between any two testers is needed if any of 
ttie conditions shown in FIG. 5 hold for TEST STEP 

35 i: In RG. 5, the fields that are marked as "null" 
indicate that input or output exchange with the 
tester and the multi-user system implementation 
must not take place. Any input and output ex- 
change that will not affect Condition 1 is marked as 

40 to signify a "don't care" condition. The vari- 
able n used in TEST STEP field of the above 
tables is defined as 1 < n < x-1 for an initial test 
sequence with x tests. 

Condition 1 corresponds to the case where 

45 TESTER A sends an input called inputj to a multi- 
user system implementation which causes TEST- 
ER B to receive an output called outputk. In this 
case, TESTER B needs to be informed to wait for 
outputk from the multi-user system implementation. 

50 In this way any. output erroneously sent by a multi- 
user system implementation to TESTER B before 
inputj is sent by TESTER A will be caught. Also, 
where an implementation is expected to generate 
outputic within a certain time, the coordination 

55 mechanism will permit such testing to occur. 

Condition 2 represents tiie case when different 
testers are to send inputs to a multi-user system 
implementation. In the tables given for Condition 2 
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and shown in FIG, 6. TESTER A receives outpu^ 
after TESTER B needs to be flagged so that it can 
send inputit to the multi-user system implementa- 
tion. Note that output] may be generated as a 
response to an input sent to the multi-user system 
implementation by any tester except TESTER B. If 
this did not occur. TESTER B would not be able to 
detennine the conrect time at which to send inputs. 

At the completion of the third step of the meth- 
od, the tests that require coordination (I.e., each 
test that satisfies Condition 1 or 2) are identified in 
each tester. 

At the fourth step, step 40, of the method, 
coordination among the testers is accomplished by 
means of a pseudo-message (or signal) called 
FLAG defined among the testers. This pseudo- 
message can be exchanged among the testers 
over the irrter-tester communication facility. FLAG 
is defined tx)th as an input and output to be ex- 
changed '^among the testers (e.g., A7FLAG or 
BIFLAG). Since it is defined as out-of-band com- 
munication and carried by a separate medium, the 
communications between testers does not affect 
the stale of the Inter-tester multi-user system im- 
plementation under test nor does it cause an input 
or output exchange to occur with the implementa- 
tion. The tests that satisfy Conditions 1 or 2 as 
defined in the previous step are coordinated by 
inserting the FLAG pseudo-message into the test- 
ers as shown in FIG: 7 for a Condition 1 occur- 
rence and in RG. 8 for a Condition 2 occurrence. 

In the above tables of RG 7, TESTER A sends 
FLAG to TESTER B before executing TEST STEP 
i. Similarly, TESTER B waits for a FLAG from 
TESTER A before it starts waiting for outputk. 
Since coordination between testers is not a part of 
the test, there is no test number assigned to the 
steps corresponding to exchanging FLAG. Also, 
there are no timing constraints on waiting for FLAG 
(i.e., no timer is running) which allow different test- 
ers with different speeds of test execution to be 
synchronized. 

In order to coordinate two testers ttiat satisfy 
Condition 2, FLAG exchanges are inserted as 
shown In FIG. 8. After receiving output|. TESTER A 
informs TESTER B to send inputu to the multi-user 
system implementation. TESTER B waits for FLAG 
from TESTER A before sending inputk- 

For the general case, each tester table includes 
the fields to represent the coordination messages 
exchanged with each tester. For example, tiie fields 
of MSG TO TESTER C, MSG FROM TESTER C 
will be added to the above coordinated tester ta- 
bles for tile case of tiiree testers (A, B, and C). If 
coordination messages are not provided, the tests 
corresponding to Conditions 1 and 2 cannot be 
tested. As a result, complete coverage of state 
transitions for the multi-user system will not be 



provided. 

Common practice in the industry, because of 
lack of a coordination mechanism, is to utilize a 
single tester system instead of a multiple tester 

5 system as described above. The remaining testers 
are replaced with standard user equipment (off-the- 
shelf equipment built to be used witti tiie multi-user 
system). This practice even furtiier limits \he test- 
ing since many unexpected or illegal message 

10 tests, in addition to tiie tests satisfying Conditions 1 
and 2, cannot be run. As briefly described at>ove, 
standard user equipment cannot generate extraor- 
dinary messages, such as unexpected and illegal 
messages. On the other hand, a tester is a system 

15 ttiat is specifically built for testing implementations 
and can easily generate non-default behavior. The 
coordination mechanism Is also easily realized 
among testers, whereas it is almost impossible to 
have among standard user equipment. 

20 

EXAMPLE 



25 Consider the example given in RG. 1 where a 
multi-user system with two users, called Userl and 
User2 is modeled as a finite state machine. This 
finite state machine represents a simplified version 
of tfie HOLD service offered by a digital commu- 

30 nication switch. At each state of the finite state 
machine of FIG. 1 , tiiere is a loopback transition to 
tiie state itself bearing an edge label shown as an 
input/output pair and labeled as 
Userl ?status/User1 lstatus_Nx 

05 where Nx can be any of the states of tiie finite 
state machine. For simplicity, tills message is not 
shown in FIG. 1. After this input/output exchange 
takes place, the finite state machine remains in the 
same state or loops back on Itself. For example, at 

40 state N1 , this pair is labeled as 
Userl ?status/User1 lstatus_N1 . 
After exchanging this input/output with Userl. ttie 
state of the finite state machine is still N1. For 
states N4 and N4_H0LD. this pair is also defined 

45 for User2 as follows 

User2?status/User2!status_Nx 
where Nx is either N4 or N4_H0LD. 

In order to illustrate the inventive method de- 
scribed above, a coordinated test sequence for two 

so testers (Note: there are two user entities) is gen- 
erated for tiie finite state machine given in FIG. 1. 

The method used to generate an initial test 
sequence for the finite state machine shown in FIG. 
1 . The initial test sequence, which has the format 

55 described earlier, is given in FIG. 9 and is contin- 
ued into FIG. 10. The initial test sequence of FlGs. 
9 and 10 requires minimum number of tests to 
completely cover the state transitions given in RG. 
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1. 

At the second step, the initial test sequence of 
FIG. 9 and 10 is divided among the two testers 
called TESTER 1 and 2 representing User1 and 
User2, respectively. The test sequence of each 5 
tester, derived from the initial test sequence of 
FIGs. 9 and 10 as described before, is given in in 
the three left-hand columns of both tables in RG. 
11. 

The tests that need coordination are identified io 
as each test of TESTER 1 and 2 is checked 
against Conditions 1 and 2. The following tests in 
TESTER 1 and 2 satisfy Condition 1: Tests 4. 14, 
16. 28. 33. 36. 40. 46. 50. For example, as Test 4. 
Tester 2 needs to be infonmed by Tester 1 in order /5 
that Tester 2 will start waiting for "setup" message 
from the multi-user system implementation under 
test. The following tests' in TESTER 1 and 2 satisfy 
Condition 2: Tests 8. 10, 18. 21. 25, 30. 47. For 
example, at Test 8, Tester 2 has to be flagged by 20 
Tester 1 to send "alert** input to the multi-user 
system implementation under test. 

At the final step, for each of the tests that are 
identified as needing coordination, FLAG messages 
are inserted appropriately. The coordinated tests 25 
for each tester is shown in FIG. 11. 

Certain l)enefits accrue from the use of the 
method described herein. For example, the time to 
run the tests Is reduced to several hours for such 
complex services from several weeks since the 30 
tester action can be automated. Testing with a 
stctndard user equipment typically requires inter- 
vention of one or more operators. Using coordi- 
nated multiple testers increases the effectiveness 
of testing multi-user systems by completely cover- 35 
ing all testable aspects of an implementation in- 
cluding under expected, unexpected and illegal 
conditions, and allows tiie automation of tests re- 
ducing the run time of tests by a several orders of 
magnitude. This method synchronizes the testers 40 
that are running at different speeds and that are 
independent of each other. This metiiod is ap- 
plicable to a wide class of implementations that 
bring services to more than one user, such as 
digital communication switches, PBXs, communica- 45 
tion protocols for layers above network layer, soft- 
ware systems and VLSI systems. The metiiod de- 
scrit>ed above provides a complete coverage in 
testing multi-user system implementations that are 
modeled as finite state machines. Without this so 
method, transitions that are corresponding to Con- 
ditions 1 and 2 cannot be tested. This method 
allows the automation of tests by using testers and, 
therefore, reduces the time to test multi-user sys- 
tem implementations by several orders of mag- 55 
nitude (for example, for complex communication 
protocols specified for multi-user systems, from 
weeks to hours). This method allows testing several 



aspects of an implementation simultaneously by 
providing coordination which makes possible the 
use of several testers simultaneously. Therefore, 
this method provides a test count reduction factor 
of up to n times for a multiple tester system with n 
testers. In this metiiod, the total number of tests for 
the complete test sequence (i.e., the sum of the 
number of tests run by each tester) is minimum 
since the initial test sequence is obtained by a 
minimum cost metiiod. This method provides a 
better observability (i.e., error detection capability) 
than any single tester metiiod used to test a multi- 
user system implementation, since all of the testers 
are simultaneously observing the behaviour of an 
implementation. 



Claims 

1. A method for testing conformance of a testable 
multi-user system to a prescribed specification, 
said multi-user system corresponding to a finite 
state machine characterized by a state diagram, 
said method comprising the steps of: 
generating a test sequence for said multi-user sys- 
tem: 

dividing said test sequence among users in order 
that a subset of said test sequence conresponding 
to each user of tiie system exists: 
identifying instances among all subsets of the test 
sequences wherein coordination between tiie users 
is required "In order for tiie subsets of the test 
sequence to be run simultaneously by the users: 
and 

inserting coordination primitives into those subsets 
of the test sequence where a need for coordination 
was identified in the previous step. 
Z The method or operation claim 1 wherein said 
finite state machine is characterized by a state 
diagram having states and original edges intercon- 
necting the states, said step of generating the test 
sequence comprising the steps oh 
evaluating unique input/output sequences for each 
state of said finite state machine so tiiat the unique 
input/output sequence selected for each state ex- 
hibits a minimum cost among all unique 
input/output sequences evaluated for that state; 
expanding said state diagram as a test graph in- 
cluding said states, said original edges, and a 
plurality of test edges interconnecting tiie states, 
said plurality of test edges for testing original 
edges of the state diagram: 
evaluating said test graph for symmetry and. if not 
symmeti'ic. duplicating selected test edges of said 
plurality to make said test graph symmetric with a 
minimum cost; and 

generating a tour of the symmetric test graph 
wherein each test edge is traversed at least once. 
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so that said tour includes said test sequence for 
said finite state machine at a minimum cost. 
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FIG.5 
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FIG.6 
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FIG.7 
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FIG. 8 
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FIG.ll 
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